Device independent text captioned telephone service

ABSTRACT

A communication system and method for displaying text captions corresponding to voice communications between an assisted user&#39;s mobile wireless device and a separate hearing user&#39;s device includes at least one communication component configured to enable the appliance to communicate with a relay, a display, and a processor operably coupled to the at least one communication component and the display. The processor is configured to enable the assisted user to establish an association between the appliance and the mobile device, receive text originating at the relay, the text corresponding to a transcript of the hearing user&#39;s voice signal originating at the hearing user&#39;s device, and cause text captions corresponding to the received text to be displayed on the display.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/988,407, filed Jan. 5, 2016, which is a continuation of U.S. patentapplication Ser. No. 14/553,291, filed Nov. 25, 2014, and issued as U.S.Pat. No. 10,015,311 on Jul. 3, 2018, which is a continuation of U.S.patent application Ser. No. 13/790,420, filed Mar. 8, 2013, and issuedas U.S. Pat. No. 8,917,821 on Dec. 23, 2014, which is a continuation ofU.S. patent application Ser. No. 12/102,057, filed Apr. 14, 2008, andissued as U.S. Pat. No. 8,416,925 on Apr. 9, 2013, which is acontinuation-in-part of U.S. patent application Ser. No. 11/391,141,filed Mar. 28, 2006, and issued as U.S. Pat. No. 7,881,441 on Feb. 1,2011, which in turn claimed the benefit of priority to U.S. ProvisionalApplication No. 60/695,099, filed Jun. 29, 2005, all of which are herebyincorporated by reference as if fully set forth herein.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable.

BACKGROUND OF THE INVENTION

The present application relates generally to telephone systems andspecifically to telephone systems that provide for real-time textcaptioning for the hard of hearing.

Modern telecommunication services provide features to assist those whoare deaf or hard of hearing. One such feature is the text telephone(TTY) also known as a telecommunication device for the deaf (TDD). TTYsallow for text communication through the telephone system by generatingtones that may be transmitted by analog telephone lines in response to auser typing letters on a keyboard.

Hearing users may communicate with deaf users who have TTY devicesthrough so-called “relays.” A relay is a service funded by telephonecommunication surcharges that provides a “call assistant” whointermediates between a deaf user and a hearing user. The call assistantcommunicates with the deaf user using a TTY and communicates with thehearing user by voice over a standard telephone line.

A relay service may also be used to help those who are not deaf, buthard of hearing, by providing a captioned telephone. With a captionedtelephone, a user who is hard of hearing may carry on a normal telephoneconversation with a hearing user while a text transcription of the wordsspoken by the hearing user is displayed on the telephone. The texttranscription allows the hard of hearing user to confirm his or herunderstanding of the spoken words. Unlike with a conventional relayservice, captioned telephone requires extremely fast text transcriptionusing a computer executed voice recognition system (typically withrevoicing by a trained operator) so that the text captioning is asnearly as possible contemporaneous with the spoken words.

Text captioned telephones of this type are marketed under the servicemark CAPTEL and described in U.S. Pat. Nos. 6,307,921 and 6,075,842assigned to the assignee of the present invention and herebyincorporated by reference.

A text captioned telephone system employs a relay service in a modewhere the relay transmits both the voice of the hearing user and a textstream of the spoken words by that user. A telephone or display deviceof the receiving hard of hearing user provides a text display fordisplaying the captions.

In a single-line text captioned telephone, text and voice signals may bereceived over a single telephone line and separated electronically. Thetext captioned telephone includes software that automatically dials therelay, and providing the relay with the telephone number of the ultimatedestination of the call. For incoming telephone calls, the hard ofhearing user must normally hang up and redial the caller through therelay.

In a two-line text captioned telephone, a first telephone linecommunicates with the caller, and a second telephone line communicatesexclusively with the relay. The two-line system allows text captioningto be easily used on incoming calls by allowing the text captionedtelephone to dial out to the relay when the call is received and forwardthe necessary voice signal.

Many users of text-captioned telephony have jobs requiring significantuse of the telephone as made possible by text-captioning. These usersface a significant problem when they need to make use of telephonesystems outside of their normal work environment where text captioningtelephones may not be available or equipment such as cellular telephonesthat does not support text captioning must be used. Many workplaces useproprietary private branch exchange (PBX) telephone lines that do notsupport analog text captioning telephones, the latter which requirestandard telephone connections.

Other problems with existing telephony systems include an inability toconfirm that a communication has been initiated by a person at a callback telephone number, system users that allow others to use theiraccounts to share or avoid costs, lack of options for terminating callsto persons that use relay systems, inability for a relay user toidentify that an incoming call is a relay call and the identity of thecaller and how to present information to an emergency call center thatcan be used by the call center to call back or establish communicationwith a relay user.

SUMMARY OF THE INVENTION

At least some embodiments include a method of commencing a captionassisted telephone call using a combination of a first party telephoneterminal and a first party Network appliance, the method comprising thesteps of (a) connecting the first party Network appliance to a relay,(b) using the first party Network appliance to specify a first partycall back telephone number, (c) using the relay to dial the first partycall back telephone number, (d) specifying a confirmation activity thatmust be performed via the first party telephone terminal in order tocontinue a telephone call process, (e) when the first party telephoneterminal is answered, monitoring activities via the first partytelephone terminal and (f) when the confirming activity occurs,continuing the telephone call process.

In some embodiments the step of specifying a confirmation activityincludes presenting instruction via the first party Network appliancespecifying confirming information to be entered via the first partytelephone terminal, the step of monitoring including monitoringinformation input via the first party telephone terminal. In some casesthe method further includes the step of also presenting information viathe first party telephone terminal indicating that the call is a captionassisted telephone call.

In some cases the confirming instruction instructs the first party topress at least one specific dual-tone multi frequency (DTMF) button onthe first party telephone terminal. In some cases the first partyNetwork appliance includes a display screen and wherein the step ofpresenting a confirmation instruction includes presenting instructionsvia the display screen.

In some cases the step of monitoring for the confirming activitycontinues until one of the confirming activity occurs and a time outperiod expires and, when the time out period expires prior to theconfirming activity occurring, the method further includes the step ofdisconnecting from the first party telephone terminal. In some cases theconfirming activity includes the first party speaking a specific phraseinto the first party telephone terminal.

Some embodiments further include the step of using the first partyNetwork appliance to specify a second party target telephone number, thestep of continuing the telephone call process including calling thesecond party target telephone number. In some cases the confirmingactivity requires the first party to enter a second party targettelephone number via at least one of the first party Network applianceand the first party telephone terminal.

In some cases the step of using the first party Network appliance tospecify a second party target telephone number includes specifying thesecond party target telephone number prior to using the relay to dialthe first party call back telephone number. In some cases the Networkappliance is a wireless device. In some cases the wireless deviceincludes the telephone associated with the first party call backtelephone number.

Some embodiments include a method of commencing a caption assistedtelephone call using a combination of a first party telephone terminaland a first party Network appliance, the method comprising the steps of(a) connecting the first party Network appliance to a relay, (b) usingthe first party Network appliance to specify a first party call backtelephone number and a second party target telephone number, (c) usingthe relay to dial the first party call back telephone number, (d)presenting a confirmation instruction to the first party via the firstparty Network appliance instructing the first party to enter confirminginformation via the first party telephone terminal, (e) monitoringinformation input via the first party telephone terminal and (f) whenthe confirming information is entered, continuing the telephone callprocess.

Some embodiments further include the step of also presenting instructionvia the first party telephone terminal indicating that the call is acaption assisted telephone call. In some cases the confirminginstruction instructs the first party to press at least one specificdual-tone multi frequency (DTMF) button on the first party telephoneterminal. In some cases the step of monitoring information continuesuntil one of the confirming activity occurs and a time out periodexpires and, when the time out period expires prior to the confirmingactivity occurring, the method further includes the step ofdisconnecting from the first party telephone terminal.

Some embodiments include a system for commencing a caption assistedtelephone call using a combination of a first party telephone terminaland a first party Network appliance, the system comprising a first partytelephone terminal, a second party telephone terminal, a first partyNetwork appliance and a relay linkable to the first and second partytelephone terminals and to the Network appliance, wherein, when thefirst party Network appliance communicates with the relay and provides afirst party call back telephone number to commence a telephone callprocess, the relay dials the first party call back telephone number,monitors activities performed via the first party telephone terminal andonly continues the telephone call process when a confirming activityoccurs.

In some cases the relay further presents confirmation instructions viathe first party Network appliance indicating a confirmation process thatshould be performed via the first party telephone terminal. In somecases the relay also presents instruction via the first party telephoneterminal indicating that the call is a caption assisted telephone call.In some cases the confirming instruction instructs the first party topress at least one specific dual-tone multi frequency (DTMF) button onthe first party telephone terminal. In some cases the first partyNetwork appliance includes a display screen and wherein the confirminginstructions are presented via the display screen. In some cases therelay monitors for confirming activity until one of the confirmingactivity occurs and a time out period expires and, when the time outperiod expires prior to the confirming activity occurring, the relaydisconnects the first party telephone terminal.

In some cases the first party Network appliance is a wireless device. Insome cases the wireless device includes the first party telephoneterminal. In some cases the first party telephone terminal and the firstparty network appliance form parts of a single communication device.

A system for presenting caller ID information related to a captionassisted telephone call, the system comprising at least a first partyNetwork appliance that includes a Network appliance display, (b) a firstparty telephone service, (b) a relay linkable to the first party Networkappliance via the Internet, the relay including a processor andprogrammed to notify the first party that a second party is attemptingto communicate with the first party by, when a second party links to therelay to commence a communication with the first party via the firstparty telephone service and the first party Network appliance,presenting both relay service identifying information and second partyidentifying information to the first party via at least one of the firstparty network appliance and the first party telephone service.

In some cases the relay presents the service identifying information viathe telephone service and presents the second party identifyinginformation via the first party Network appliance. In some cases thefirst party is associated with a first party telephone terminal thatincludes a telephone display that is separate from the first partyNetwork appliance display, the relay presenting the service identifyinginformation via the first party telephone terminal display. In somecases the first party Network appliance is also used by the first partyas a first party telephone terminal and wherein the relay presents thesecond party identifying information and also presents the serviceidentifying information via the Network appliance display.

Some embodiments include a system for presenting caller ID informationrelated to a caption assisted telephone call, the system comprising (a)at least a first party Network appliance that includes a Networkappliance display, (b) a first party telephone terminal including atelephone terminal display, (c) a relay linkable to the first partyNetwork appliance via the Internet, the relay including a processor andprogrammed to notify the first party that a second party is attemptingto communicate with the first party by, when a second party links to therelay to commence a communication with the first party via the firstparty telephone terminal and the first party Network appliance,presenting service identifying information and second party identifyinginformation to the first party via the telephone terminal display andthe Network appliance display, respectively.

Some embodiments include a method for presenting caller ID informationrelated to a caption assisted telephone call, the method for use with afirst party Network appliance that includes a Network appliance display,a first party telephone service and a relay linkable to the first partyNetwork appliance via the Internet, the relay including a processor, themethod comprising the steps of, when a second party links to the relayto commence a communication with the first party via the first partytelephone service and the first party Network appliance, (i) presentingrelay service identifying information to the first party via at leastone of the first party network appliance and the first party telephoneservice and (ii) presenting second party identifying information to thefirst party via at least one of the first party network appliance andthe first party telephone service.

In some cases the step of presenting relay service identifyinginformation to the first party includes presenting the relay serviceidentifying information via the telephone service and the step ofpresent second party identifying information includes presenting thesecond party identifying information via the first party Networkappliance display screen. In some cases the first party is associatedwith a first party telephone terminal that includes a telephone displaythat is separate from the first party Network appliance display, therelay presenting the service identifying information via the first partytelephone terminal display.

In some cases the first party Network appliance is also used by thefirst party as a first party telephone terminal and wherein the relaypresents the second party identifying information and also presents theservice identifying information via the Network appliance display.

Some embodiments include a method for facilitating an emergency voicecall using a first party Network appliance that communicates via anInternet protocol (IP) and that has an IP address, the method comprisingthe steps of (a) connecting the first party Network appliance to arelay, (b) using the first party Network appliance to specify a targettelephone number, (c) recognizing the target telephone number as anemergency telephone number, (d) assigning a temporary telephone numberto the first party where calls to the temporary telephone number arerouted to the relay and (e) storing the temporary telephone number withthe IP address of the first party Network appliance in a databaseaccessible by the relay.

Some embodiments further include the steps of using the emergencytelephone number to establish a first communication with an emergencyservice provider and presenting the temporary telephone number to theemergency service provider. Some embodiments further include the stepsof, subsequent to the first communication, the emergency serviceprovider using the temporary telephone number to link to the relay andcommence a second communication, when the service provider links to therelay, the relay using the temporary telephone number to identify the IPaddress of the first party Network appliance and establishingcommunication with the first party via the first party Networkappliance. In some cases the step of storing further includes storing acall back telephone number that is associated with the first party withthe temporary telephone number and the IP address of the first partyNetwork appliance.

In some cases, when the service provider links to the relay, the relayfurther uses the temporary telephone number to identify the call backnumber associated with the first party and further establishescommunication with the first party via the call back telephone number.In some cases the temporary telephone number is unassigned to the firstparty after expiration of a preset period. Some embodiments furtherinclude the step of , subsequent to expiration of the preset period,assigning the temporary telephone number to second party that uses asecond party Network appliance to specify an emergency telephone number.

Some embodiments include a method for facilitating an emergency voicecall using a first party Network appliance that communicates via anInternet protocol (IP) and that has an IP address, the method comprisingthe steps of (a) connecting the first party Network appliance to a relay(b) using the first party Network appliance to specify a targettelephone number, (c) recognizing the target telephone number as anemergency telephone number, (d) assigning a temporary telephone numberto the first party where calls to the temporary telephone number arerouted to the relay, (e) storing the temporary telephone number with theIP address of the first party Network appliance and a first party callback number usable to place a telephone call to the first party in adatabase accessible by the relay, (f) using the emergency telephonenumber to establish a first communication with an emergency serviceprovider and presenting the temporary telephone number to the emergencyservice provider.

Some embodiments further include the steps of, subsequent to the firstcommunication, the emergency service provider using the temporarytelephone number to link to the relay and commence a secondcommunication, when the service provider links to the relay, the relayusing the temporary telephone number to identify the IP address of thefirst party Network appliance and the first party call back number andestablishing communication with the first party via the first partyNetwork appliance and the first party call back telephone number.

Some embodiments include a method for facilitating a voice call using afirst party Network appliance that communicates via an Internet protocol(IP) and that has an IP address, the method comprising the steps of (a)connecting the first party Network appliance to a relay, (b) using thefirst party Network appliance to specify a target telephone number, (c)assigning a temporary telephone number to the first party where calls tothe temporary telephone number are routed to the relay and (e) storingthe temporary telephone number with the IP address of the first partyNetwork appliance in a database accessible by the relay.

Some embodiments further include the steps of using the telephone numberto establish a first communication with a second party and presentingthe temporary telephone number to the second party. Some embodimentsfurther include the step of, subsequent to the first communication, thesecond party using the temporary telephone number to link to the relayand commence a second communication, when the second party links to therelay, the relay using the temporary telephone number to identify the IPaddress of the first party Network appliance and establishingcommunication with the first party via the first party Networkappliance. In some cases the step of storing further includes storing acall back telephone number that is associated with the first party withthe temporary telephone number and the IP address of the first partyNetwork appliance.

In some cases, when the second party links to the relay, the relayfurther uses the temporary telephone number to identify the call backnumber associated with the first party and further establishescommunication with the first party via the call back telephone number.In some cases the temporary telephone number is unassigned to the firstparty after expiration of a preset period. Some embodiments furtherinclude the step of determining that the target telephone numbercorresponds to an emergency service provider and wherein the step ofassigning a temporary telephone number to the first party includesassigning a temporary telephone number only when the target telephonenumber corresponds to the an emergency service provider.

Some embodiments include a telephone call captioning system comprising afirst party Network appliance linked to a network for communicating viaan Internet protocol (IP) and having a first party IP address, a firstparty telephone associated with a first party call back telephonenumber, a second party telephone associated with a second partytelephone number, a database, a relay linked to the Network and thedatabase, the relay including a processor and programmed to perform thesteps of, when the first party Network appliance is linked to the relayto commence a communication with a second party (i) receiving a targettelephone number associated with a second party, (ii) assigning atemporary telephone number to the first party where calls to thetemporary telephone number are routed to the relay and storing thetemporary telephone number with the IP address of the first partyNetwork appliance and the first party telephone number in the databasefor subsequent access by the relay.

In some cases the relay processor is further programmed to perform thesteps of using the emergency telephone number to establish a firstcommunication with a second party and presenting the temporary telephonenumber to the second party. Some embodiments further include the stepsof, subsequent to the first communication, the second party using thetemporary telephone number to link to the relay and commence a secondcommunication, when the second party links to the relay, the relay usingthe temporary telephone number to identify the IP address of the firstparty Network appliance and the first party call back number andestablishing communication with the first party via the first partyNetwork appliance and the call back number. In some cases the relayprocessor unassigns the temporary telephone number from the first partyafter expiration of a preset period.

Some embodiments include a method for use with an Internet appliancethat is linkable to a call assist relay via the Internet to facilitatecommunication with a first party via the Internet appliance, theInternet appliance communicating via an Internet protocol (IP), themethod for facilitating an activity when a communication is attemptedwith the Internet appliance and the Internet appliance is de-linked fromthe relay, the method comprising the steps of providing a database thatis accessible to the relay, the database including instructionsregarding activities to be performed when a communication is attemptedwith the Internet appliance and the Internet appliance is de-linked fromthe relay, monitoring communications at the relay and, when a secondparty attempts to communicate with the Internet appliance (i)determining that the Internet appliance is de-linked from the relay and(ii) commencing a terminating action related to the communication.

In some cases the step of commencing a terminating activity includespresenting an audible message to the second party indicating that thecommunication cannot be completed. In some cases the step of commencinga terminating activity further includes receiving a spoken message fromthe second party and commencing transcription of the spoken message intoa text message. Some embodiments further include the step of, at a timesubsequent to the transcription when the first party links to the relay,presenting an indication that a transcribed message exists for the firstparty and allowing the first party to view the text message. In somecases the first party is also associated with a first party call backtelephone number, the database also storing the first party call backtelephone number, the step of commencing a terminating action includingaccessing the first party call back telephone number in the database,calling the first party call back telephone number and presentinginformation to the first party via a first party telephone indicatingthat a communication has been attempted.

Some embodiments further include the steps of, when the first partytelephone remains unanswered for a time out period, receiving a spokenmessage from the second party and commencing transcription of the spokenmessage into a text message. In some cases the step of providing adatabase includes the steps of, via the Internet appliance, providing aninterface screen to the first user including a plurality of differentpossible terminating activities, receiving at least one terminatingactivity selection via the interface screen and storing the terminatingactivity selection for subsequent use. In some cases the first party isalso associated with additional contact information for electronicallycommunicating with the first party, the database also storing theadditional contact information associated with the first party, the stepof commencing a terminating action including accessing the additionalcontact information associated with the first party in the database,attempting to link to the first party via the additional contactinformation and presenting information to the first party indicatingthat a communication has been attempted via the first party Internetappliance.

Still other embodiments include a method of controlling use of a captionassisted telephone call relay by a first party, the first party using acombination of a first party telephone terminal and a first partyNetwork appliance to facilitate communication, the first party Networkappliance communicating via an Internet protocol (IP) and having an IPaddress, the method comprising the steps of (a) prescribing a singlefirst party call back number for the first party to communicate via therelay, (b) storing the first party call back number and the first partyNetwork appliance IP address in a database accessible to the relay, (c)connecting the first party Network appliance to the relay, (d)specifying a target telephone number associated with a second party, (e)via the relay (i) identifying the first party Network appliance IPaddress, (ii) accessing the database and identifying the first partycall back number and (iii) using the first party call back number toestablish a communication link between the first and second parties.

Other embodiments include a system for facilitating an activity when acommunication is attempted and unsuccessful in a call assistenvironment, the system comprising a first party Network appliancelinked to a network to communicate via an Internet protocol (IP) andhaving an IP address, a relay linked to the network and usable tocommence a communication with the first party Network appliance, therelay including a processor that runs a program to perform the steps ofproviding an interface to a first party via the first party Networkappliance enabling the first party to select one of a plurality ofdifferent options for activities to be performed when a communication isattempted with the first party Network appliance when the first partyNetwork appliance is de-linked from the relay, accepting the first partyselection and storing the selection in the database as instructionsregarding activities to be performed when a communication is attemptedwith the Network appliance and the Network appliance is de-linked fromthe relay, monitoring communications at the relay and, when a secondparty attempts to communicate with the first party Network appliance (i)determining that the Network appliance is de-linked from the relay and(ii) accessing the database and identifying the first party selectionand (iii) commencing the first party selection to terminate thecommunication.

In some cases the first party selection includes presenting an audiblemessage to the second party indicating that the communication cannot becompleted. In some cases the first party selection includes receiving aspoken message from the second party and commencing transcription of thespoken message into a text message. In some cases the processor furtherruns the program to perform the step of, at a time subsequent to thetranscription when the first party links to the relay, presenting anindication that a transcribed message exists for the first party andallowing the first party to view the text message. In some cases thefirst party is also associated with a first party call back telephonenumber, the database also storing the first party call back telephonenumber, the first party selection including commencing a terminatingaction by accessing the first party call back telephone number in thedatabase, calling the first party call back telephone number andpresenting information to the first party via a first party telephoneindicating that a communication has been attempted.

In some cases the processor further performs the steps of, when thefirst party telephone remains unanswered for a time out period,receiving a spoken message from the second party and commencingtranscription of the spoken message into a text message. In some casesthe first party is also associated with additional contact informationfor electronically communicating with the first party, the database alsostoring the additional contact information associated with the firstparty, the first party selection including commencing a terminatingaction by accessing the additional contact information associated withthe first party in the database, attempting to link to the first partyvia the additional contact information and presenting information to thefirst party indicating that a communication has been attempted via thefirst party Internet appliance.

Other embodiments include a method of commencing a caption assistedtelephone call using a combination of a first party telephone terminaland a first party Network appliance, the method comprising the steps of(a) connecting the first party Network appliance to a relay, (b) usingthe first party Network appliance to specify a first party call backtelephone number, (c) using the relay to dial the first party call backtelephone number, (d) using at least one of the first party telephoneand the first party Network appliance to specify a confirmation activitythat must be performed in order to continue a telephone call process,where the first party telephone is used to specify the confirmationactivity, the confirmation activity having to be performed using atleast one of the first party telephone terminal and the first partyNetwork appliance and where the first party Network appliance is used tospecify the confirmation activity, the confirmation activity having tobe performed using at least the first party telephone, (e) when thefirst party telephone terminal is answered, monitoring activities via atleast one of the first party telephone terminal and the first partyNetwork appliance and (f) when the confirming activity occurs,continuing the telephone call process.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a representation of a first embodiment of the inventionproviding text-captioned telephony using a PBX office telephone andInternet-connected computer;

FIG. 2 is a perspective view of a voice-tap connecting the PBX telephoneof FIG. 1 to the computer to allow transfer of voice signals over theInternet to a relay;

FIG. 3 is a caller interface window as may be displayed on the computerof FIG. 1 allowing on-demand captioning of the telephone call;

FIG. 4 is a flowchart of the principle steps of a program executed onthe computer of FIG. 1 when captioning is demanded by the caller;

FIG. 5 is a figure similar to that of FIG. 1, showing an alternativeembodiment in which the voice signal is routed through a standardtelephone line to the relay and the text is returned to the computer;

FIG. 6 is a figure similar to that of FIG. 4 showing the flowchart forthe embodiment of FIG. 5;

FIG. 7 is a figure similar to that of FIG. 3 showing a caller interfacewindow for entering a PIN number to link captioning text to a particularcall;

FIG. 8 is a figure similar to that of FIGS. 1 and 5 showingimplementation of a text-captioning using VOIP transmissions;

FIG. 9 is a figure similar to that of FIGS. 7 and 3 showing a callerinterface window for a VOIP telephone implemented on a standardcomputer;

FIG. 10 is a flowchart of the principal steps executed by the computerin implementing the text-captioned telephony using VOIP telephones;

FIG. 11 is a fragmentary view of the computer of FIG. 8 showing theaddition of a video camera so that voice and video may be transmitted tothe other caller;

FIG. 12 is a fragmentary view of the caller interface window of FIG. 9showing a video inset possible with the configuration of FIG. 11;

FIG. 13 is a flow chart illustrating an exemplary call confirmationprocess that is consistent with at least some aspects of the presentinvention;

FIG. 14 is a schematic illustrating an exemplary browser page that maybe presented during the process shown in FIG. 13;

FIG. 15 is a flow chart illustrating a flexible caller ID feature thatis consistent with at least some embodiments of the present invention;

FIG. 16 is a schematic illustrating a browser page that may be presentedduring one of the process steps shown in FIG. 15;

FIG. 17 is a schematic illustrating a telephone including a caller IDdisplay and information displayed on the display during one of theprocess steps in FIG. 15;

FIG. 18 is similar to FIG. 17, albeit showing a browser page displayedon a telephone/computer device and a caller ID window or displaypresented over the browser page;

FIG. 19 is a flow chart illustrating a call back feature that isconsistent with at least some aspects of the present invention;

FIG. 20 is a flow chart illustrating a terminating feature set upprocess as consistent with at least some aspects of the presentinvention; and

FIG. 21 is a flow chart illustrating implementation of a terminatingfeature that is consistent with at least some inventive embodiments.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to FIG. 1, in a first embodiment of the invention, a textcaptioned telephone system 10 may be implemented within a typical officehaving a PBX telephone terminal 12 and a desk top computer 14. The desktop computer 14 includes a display screen 16, a base unit 18 (includinga processor, memory, disk drives and importantly a sound card) and akeyboard or other entry device 20.

As is understood in the art, the PBX telephone terminal 12 may beconnected through an office wall jack 22 (not labeled in FIG. 1) to aPBX network 24 using a proprietary communication protocol. The PBXnetwork 24 communicates with a public switched telephone network 26 thatallows the PBX telephone terminal 12 to connect to an external telephoneterminal 28 to originate a call to the external telephone terminal 28 orto receive a call from the external telephone terminal 28.

The computer 14 may connect through an Ethernet card to an Ethernetnetwork 30, passing to a local router 32 to connect to the Internet 34or may use one of a number of well known wireless standards to connectto the local router 32.

Referring now also to FIG. 2, a voice signal tap 36 has a short cablestub 38 terminating in an RJ-12 jack 40 that may be received in thehandset jack for the main body of the PBX telephone terminal 12. The tap36 also provides a receiving socket for the RJ-12 jack 42 associatedwith the handset cord 44 so that the tap 36 may be simply interposedbetween the handset 46 and the main body of the PBX telephone terminal12 to conduct voice signals as analog audio signals there through.

A tap line 48 extending from the tap 36 terminates in a subminiaturephone jack 50 that may be received by audio input of the sound card ofthe base unit 18. The tap line conducts a portion of the voice signalsfrom the handset 46 to the sound card of the base unit 18.

The voice signals received by the computer 14 may be processed by aninternal program of a type well known in the art to encode the voicesignals as VOIP signals 52 that may be sent over the Internet to aserver 54 associated with a relay service 56. Programs for transmittingVOIP signal are commercially available from Skype of Luxembourg andVonage of New Jersey, USA. At the relay service 56, the encoded voicesignals may be converted back to an audio signal for transcription by acall assistant who produces corresponding captioning text 57 that may berelayed through conventional Internet transfer protocols back to thecomputer 14 where the text may be displayed.

In an alternative embodiment, a headset microphone and earpiece (notshown) may communicate separately with left and right sound cardchannels. This, along with echo canceling software in the computer 14allows the relay service 56 to separate the voice of the caption userfrom the other party for improved transcription and anonymity.

The relay service 56 may provide for human operators working with speechrecognition engines to rapidly translate voice signals into textstreams. The operation of such a relay is described in more detail inU.S. Pat. No. 6,567,503, assigned to the same assignee as the presentinvention and hereby incorporated by reference.

By placing the tap 36 in the path of the handset 46, analog audiosignals may be obtained, greatly simplifying the acquisition of theaudio signal without the need to contend with the PBX standard or theneed or awkward or unfamiliar computer associated microphones.

Referring now to FIG. 3, an office user wishing to avail themselves oftext captioning may start a text-captioning program 58 on the computer14 to provide a caller interface window 60 on the display screen 16. Thecaller interface window 60 may provide simple mouse or keyboard operatedcontrols including a caption button 62 that may be actuated by thecaller to begin the program's operation.

Referring to FIG. 4, upon pressing of the caption button 62, asindicated by process block 64, the computer 14 may initiate an Internetconnection to the relay service 56 by invoking a stored URL of the relayserver 54. At this time, the computer 14 may provide some information tothe relay including a serial number of the caller to validate thecaller's location for the purpose of identifying the particular publicauthority responsible for the relay call. The caller can be required toregister the program with a relay service database and the serial numbercan be used to authorize their use of the service, thus limitingfraudulent use of the relay service and allowing for local 911 serviceby providing information about the geographic location of the user.

As indicated by process block 66, the program 58 may then forward thetapped voice signals as VOIP signals to the relay server 54, and receivetext as indicated by process block 68 which may be displayed in text box70 of the caller interface window 60. The text box 70 includesconventional scroll-type controls 73 allowing text to be reviewed afterit has been received. The program 58 may also provide for normal fileoperations 75 including saving of text files, e-mailing text files, andthe like.

Upon completion of the call, the caller may press the caption button 62again to disconnect the call as indicated by process block 71. Thecaption button 62 may include an animation visually indicating its stateas being depressed or released or may change its label from “caption” to“end caption” indicating its changing function.

Importantly, this system allows for convenient and intuitive voicecommunication between a PBX telephone terminal 12 and telephone terminal28, either for making outgoing calls or ingoing calls, while allowingeither type of call to be captioned on demand without interruption ofthe telephone call or the need for specialized telephone equipment. Thissystem will also work without a PBX exchange and can work for a varietyof different telephone types not intended for text captioning.

Referring now to FIG. 5, in a second embodiment, a standard desk topcomputer 14 may work in conjunction with a standard telephone (i.e., nottext captioned), in this case a cell telephone 72, the latter of whichcommunicates through a cellular service 74 with the public switchedtelephone network 26, without a direct connection between the celltelephone 72 and the computer 14.

In this embodiment, the caller using the cell telephone 72 first callsthe relay service 56 as indicated by process block 76 of FIG. 6. Therelay service 56, per standard practice, receives telephone calls overthe public switched telephone network 26. The relay service 56 mayprovide an automated answering system that requests a telephone numberfrom the caller of the ultimate destination of the telephone call, inthis case, telephone terminal 28. As indicated by process block 78 ofFIG. 6, the caller may enter the requested telephone number by using thecaller's keypad on the cell telephone 72.

In the case where the relay service remains anonymous, in the sense thatthe call assistant does not speak directly with the caller, theanswering system may provide the caller with a personal identificationnumber (PIN) as indicated in process block 80. The relay service 56links the caller's particular incoming line to the PIN in an internaldatabase 57.

The caller, using a conventional browser on the desk top computer 14,then enters the URL of the server 54 of the relay service 56, asindicated by process block 82 of FIG. 6. This may also be done beforeinitiating the call to the relay service 56.

As shown in FIG. 7, the relay server 54 provides the caller with webpage86 offering a text box 88 allowing entry of the PIN received at processblock 80. The caller, by entering that PIN and press a captioninginitiation button 90, causes the relay service 56 to complete the callby connecting the incoming audio from the cell telephone 72 to thetelephone terminal 28. The relay service 56 then forwards captioningtext 55 over the Internet 34 to the caller through a text box 92 asindicated by process block 94. The text box 92 may use, for example, abrowser plug-in or instant messaging program to provide for a consistentupdating of the text as it is received, but otherwise requires nospecialized software on computer 14.

Alternatively and preferably, the caller may first contact the webpage86 using the computer 14 to provide the relay server 54 with both thenumber the caller wishes to dial and also the caller's own telephonenumber. The relay server 54 then calls the caller over the cell phone72. Once a connection is established with captions, the relay server 54dials to the other party. This automatically links the IP address to thetelephone connections without the need for PIN. The phone number of thecaller may be entered on the webpage 86 in a text box (not illustrated)and the destination phone number may be entered in another text box (notillustrated) similar to those described for the PIN.

The use of a web page and browser plug in to avoid the need forspecialized software to be on the desktop computer 14 is also applicableto the earlier embodiment of FIG. 1.

At the end of the call as indicated by process block 96, the caller maypress the captioning initiation button 90 again to terminate the call.The captioning initiation button 90 may reflect this new purpose, ofterminating the captioning, by changing its label.

Again, in this embodiment, no specialized text captioning equipment isrequired, but the system makes use of commonly available office andtelephone equipment to provide for text captioning that is not limitedto a particular location, but which may be used in any location whereInternet and telephone access may be had. In this regard, computer 14may, for example, be any Internet appliance, for example, a wirelessmobile laptop or the like. Thus a caller may work from a hot spot usinga cell telephone and a wireless laptop to obtain competent textcaptioning.

In a variation on this embodiment, set-up of the call (e.g. providingthe relay service 56 with the destination phone number) may beaccomplished using the desktop computer 14 rather than via telephone 72.When the caller calls the relay service 56, the caller enters a pinnumber to connect the call to the captioning or the connection may bemade by linking the callers phone number with a number previouslyentered on the computer 14.

In this embodiment, identification of the particular governmental entityresponsible for reimbursement for the captioning can be obtained fromthe information of the public switched telephone network 26 per standardpractice.

In a variation on this embodiment, new 3G cellular services allow theuse of both cellular voice and data connections simultaneously using acellular phone. In this case, the browser on a desktop computer 14 maybe replaced with a browser on the cell phone which provides a telephoneand Internet appliance, all in one.

Referring now to FIG. 8 in a third embodiment, a single advancedInternet appliance may be used to replace the need for a telephone. ThatInternet appliance, for example, may be a standard computer 14 equippedwith a microphone and headphone assembly 100 for making VOIP calls or aVOIP telephone 102 providing for display capabilities on screen 104.Many PBX systems now allow for the connection of VOIP phones.

Referring also to FIGS. 9 and 10, in a first variation on thisembodiment, a program 108 executed by the computer 14 provides a callerinterface window 110 on the display screen 16 of the computer allowingthe initiation of a VOIP connection as indicated by process block 112 ofFIG. 10. The caller interface window 110 provides a standard virtualtelephone keypad 114 by which a telephone number of a telephone terminal28 on the public switched telephone network 26 may be entered anddisplayed in text box 116. Pressing of a call button 120 initiates aVOIP telephone call. The basic process of initiating a VOIP call is wellunderstood in the art.

In the present invention, however, at any time, a captioning initiationbutton 122 may be pressed, causing initiation of a second VOIP telephonecall directly to the relay service 56 through the relay server 54 asindicated by process block 124. At this time, the VOIP signals 52exchanged between the computer 14 and telephone terminal 28 (optionallyvia the public switched telephone network 26) may be multicast to theserver 54 to be interpreted as voice signals by the relay service 56.The server 54 may also receive a serial number 125 identifying thecaller and assisting in allocation of relay service fees amonggovernmental entities. The relay service 56 returns captioning text 56transcribing the VOIP signals 52 back to the computer 14 to be displayedin text box 92 as indicated by process block 126.

The caller interface window 110 may also provide a volume meter 130aiding the caller in speaking to obtain sufficient signal strength fortransmission to telephone 28.

Alternatively, the call may be initiated as a captioned call by pressingthe captioning initiation button 122, then dialing the telephone number,in which case, setup information (e.g., the destination telephonenumber) together with a serial number, are forwarded to the relayservice 56 which may then initiate the VOIP call to the telephoneterminal 28 patching through signals received from computer 14 to thetelephone terminal 28.

As indicated by process block 132, the call may be terminated bypressing the call button 120 and/or captioning may be terminated bypressing the captioning initiation button 122.

When the computer 14 is used, a connection to the public switchedtelephone network 26 may also be provided for interception of emergencycalls, for example, to 911, routing those through the publicly-switchedtelephone network rather than through the Internet so as to provide forthe benefits of rapid identification of geographic location of thecaller inherent in calls made through the publicly-switched telephonenetwork and not always provided currently with the VOIP services.

Referring again to FIG. 8, the identical steps may be accomplished by aspecially programmed VOIP telephone 102, which requires only amodification in software to implement the same functions as thosedescribed above. The captioning initiation button in this case can beimplemented in software keystroke combinations without the need foradditional switches with the standard screen 104 on the VOIP telephoneserving as the text display and optional volume meter.

General purpose Internet appliances such as computer 14 make it possibleto expand the previous embodiments to the addition of a video camera 140providing video signals 142 to the Internet as well as the VOIP signals52 and the receipt of captioning text 55. Referring to FIG. 12, theaddition of video signals allows a video inset box 144 to be added tothe caller interface window 136, allowing viewing of the other party tothe conversation or the relay operator also having an Internet appliancesuch as a computer 14. This embodiment allows face-to-faceconversations, but also may allow, for example, the use of American SignLanguage or lip reading between users together with the captioningprovided by the text box 92. In other respects, the caller interfacewindow 136 may be identical to caller interface window 110 describedabove.

In a variation on this embodiment, a standard videophone may be used asmay be connected over a broadband Internet connection (or corporate LAN)to provide video and voice. Videophones are popular with both hearingpeople and deaf people that use sign language and provide a built incamera and display screen for the picture of the other party.

In this embodiment, the videophone user or video conference system usercan connect with the captioning service website and the service can sendthe captions not as IP text but as an IP video signal formatted to becompatible with the videophone. The captions are transmitted instead ofthe image of the caller, as video images of letters using standardcaptioning software that converts text to a video signals or as overlaidon the image of one caller using a video merging technique. In the casewhere a videophone provides a separate text display, this canalternatively be used for the captioning, as described above with theInternet phone, with the text being transmitted as IP text, that is,character data such as ASCII. Alternatively, IP text may be converted atthe videophone into image data to be displayed on a video screen as acaption. When IP text is received, the videophone may elect how the textis to be displayed, either as captions on a video image or as text withno other image. The IP text may be uniquely marked to identify it to thevideophone.

Additional Features

In addition to the various embodiments and aspects described above,additional features and aspects of the relay assisted systems describedabove are contemplated. A first additional feature is referred togenerally herein as a call confirmation process. To this end, asdescribed above, in at least some embodiments, when placing a relayassisted call, a hearing impaired user first opens her browser and linksto the relay server, logs into a service provider web page (see 86 inFIG. 7) and provides the relay server 54 (see FIG. 5) with both thenumber the caller wishes to dial and also the callers own telephonenumber. The relay server then calls the hearing impaired user giving theuser a prompt on her browser screen to answer the user's phone. When theuser answers the arriving call, the relay server automatically placesthe call to the second party (i.e., to the caller that the user wishesto dial). The relay 54 then conferences the two telephone connectionstogether and the call begins. At this point the relay servicetranscribes words spoken by the second party and the text captions aresent to the hearing impaired user's browser screen.

In this type system, there is a concern that someone could abuse thesystem in the following way. An unscrupulous third party could log intoa web page provided by a relay server 54, enter a call back telephonenumber that is not associated with the third party (i.e., a numberassociated with some other unsuspecting party), enter the second phonenumber that does not belong to the third party and cause the relay toconnect two unsuspecting parties. In this case, the relay server 54would dial the first telephone number and when an unsuspecting personanswers that call, the server 54 would make the call to the secondtelephone number and hence to the second unsuspecting party. When thesecond telephone is answered, at least one and potentially both of thepeople connected by the call would believe that the other person calledthem when in fact neither party placed the call.

The inventive call confirmation process has been developed to preventunscrupulous third parties from linking unsuspecting other parties usinga relay server 54 as described above. To this end, in general, theinventive call confirmation process proceeds as follows. After a hearingimpaired user contacts a web page to commence a call activity andprovides the callers own telephone number, the relay server 54 dials thecallers own telephone number and, generally simultaneously therewith,displays a message on the user's browser screen instructing the user toanswer the user's telephone and, thereafter, to perform someconfirmation activity to indicate that the user in fact intended tocommence the associated communication. For example, exemplaryconfirmation activities may include pressing one or a specific series ofdual-tone multi-frequency (DTMF) buttons on their telephone. As anotherexample, the confirmation activity may require the user to speak a wordor a specific phrase into the user's telephone receiver. Here, the relayserver 54 would be programmed so that it would not dial a second partyunless the instructed confirmation activity is performed using thetelephone associated with the call back telephone number. Thus, forexample, if the party answering the first telephone (i.e., the telephoneassociated with the call back telephone number) does not have thecaptioning screen in front of her, she will not know what confirmationactivity is required to continue the call and the relay server canterminate the call to eliminate the possibility of connecting twounsuspecting parties.

Referring now to FIG. 13, an exemplary call confirmation process 158 isshown in flow chart form. Referring also to the exemplary system shownin FIG. 5, at block 160, a first party (e.g., a hearing impaired systemuser) uses a network appliance 14 to link via network 34 to a relay 54server. Here, the network appliance may be any type of computer capableof running a browser or the like for providing web pages including apersonal computer, a laptop computer, a palm type computing device, atelephone, etc. Hereinafter, although many types of network appliancesare contemplated, the term computer will be used generally to simplifythis explanation unless indicated otherwise.

Referring still to FIGS. 5 and 13, at block 162, the first party usesthe computer to specify a first party call back telephone number as wellas a second party target telephone number (e.g., the telephone numberassociated with a second party that the first party wishes to dial). Atblock 164, relay server 54 dials the first party call back number. Inthe example of FIG. 5, the first party call back number corresponds totelephone 72. At block 168, relay 54 monitors first telephone activityto determine when the telephone 72 is answered. Once the first partytelephone 72 is answered control passes to block 170.

In at least some embodiments, at block 170, when telephone 72 isanswered, relay 54 presents information via telephone 72 indicating thatthe current call is an assisted call. This assisted call warning isprovided to help an unsuspecting person that did not intend to commencethe communication to recognize that the communication was in error andshould be terminated. For example, the assisted call warning mayindicate “This is an assisted call. If you did not commence this callactivity please hang up your telephone now.”

Next, at block 172, relay 54 presents confirmation activity instructionsvia a web page presented on the first party's computer 14 indicatingconfirmation activity that must be performed via the first party'stelephone 72 to continue the call process. Referring also to FIG. 14, anexemplary web page 150 for presenting confirmation activity instructionsis shown. Web page 150 includes instructions 154 and a code field 156.Exemplary instructions 154 instruct the user to “Please answer yourtelephone and enter the following 4-digit code via you telephone keypadto continue this call:”. A randomly generated 4-digit code is presentedin field 156 which is to be entered, in this example, to continue thecall.

Referring yet again to FIGS. 5 and 13, at block 174, after the firstparty's telephone 72 is answered, in at least some embodiments, relay 54may start a timer. Here, the relay timer is intended to time-out aperiod during which it is expected that the first party would be able tocomplete the confirmation activity required to continue the call. Forinstance, the time-out period may, in some cases, be 30 seconds. Here,where an unsuspecting user is confused by the call activity and does notterminate a call, it is contemplated that the relay 54 would terminatethe call automatically at the end of the time-out period.

At block 176, relay 54 monitors activities performed by the firstparty's telephone 72 to identify when the confirmation activity hasoccurred. In the present example, consistent with FIG. 14, the relay 54would monitor to determine when the 4-digit code in field 156 has beenentered via the first party's telephone 72. Where the confirmationactivity has not occurred, control passes to block 180 where relay 54determines whether or not the relay timer period is equal to thetime-out period. Where the relay timer period does not equal thetime-out period, control passes back up to block 176 where monitoringcontinue for the confirmation activity. Once the timer period equals thetime-out period at block 180, control passes to block 182 where relay 54terminates the call.

Referring once again to block 176, where the confirmation activityoccurs, control passes to block 178 where relay 54 calls a second partytarget telephone number and the call continues at block 179.

In at least some embodiments it is contemplated that all relay users maybe assigned a unique pass code or PIN number. Where a user has a PINnumber, referring again to FIG. 13, at block 172 the relay 54 mayprovide instructions for the user to simply enter the user's PIN numbervia the telephone keypad and, subsequently entry of the PIN number wouldbe used as confirmation activity at block 176.

In still other embodiments the confirmation activity instructions may bepresented via the first party's telephone 72 (i.e., the call backtelephone) and may require the first party to perform some confirmationactivity using the first party's computer 14. For example, theconfirmation activity instructions delivered via telephone 72 mayinstruct the user to “Enter the following 4-digit code via your browserto continue this call, 4397.” In this case, relay 54 would provide acode entry field via computer 14 and would monitor the code enteredtherein for confirmation activity. In this case, where an unsuspectinguser receives a call via telephone 72, the user would recognize that thecommunication is in error and would not participate by entering theconfirmation code via the computer 14. Here, after the time-out periodthe communication would be automatically terminated. Note that where auser cannot hear (e.g., in most cases a captioned telephone user ishearing impaired), a voice prompt on the telephone may not be usable tocause the user to enter something on their browser screen and thisembodiment would not work well.

In yet other embodiments, the confirmation activity may require entry ofa second party's target telephone number to continue a communication. Tothis end, referring again to FIG. 13, at block 162 the first party maysimply enter the first party's call back telephone number and may not berequired to enter the second party's target telephone number. Then, atblock 172, relay 54 may simply require that the first party enter thesecond party's target telephone number. Here, presumably, if the firstparty called back on telephone 72 is an unsuspecting party, whenrequested to enter a target telephone number to be called, the firstparty will recognize that the communication is in error and wouldterminate the call. However, where the first party answering telephone72 initiated the communication, the first party would be ready to enterthe second party's target telephone number. In this case, referringstill to FIG. 13, at block 176, instead of monitoring for confirmationactivity, relay 54 would simply monitor for an entered second partytarget telephone number. Where the second party's telephone number isentered, control passes to block 178 and then to block 179 to continuethe telephone call.

In still other embodiments, confirmation activity instructions may beprovided via a first party telephone that require activity using thefirst party's telephone where the instructions are provided along withan assisted call warning. Here, when a first party answers telephone 72(see again FIG. 5) the assisted call warning should be sufficient forthe first party to recognize whether or not the communication is inerror and, where the communication is in error, the first party couldterminate the call and not complete the confirmation activity. Where thefirst party initiated the communication, the first party could completethe confirmation activity via the first party telephone and continue thecommunication.

Thus, it should be appreciated that many different confirmationactivities are contemplated and ways of facilitating completion ofconfirmation activities are contemplated where a system user receivesconfirmation instructions via at least one of the user's computer ortelephone and is required to carry out confirmation activities via atleast one of the user's computer or telephone where at least one of thedevices used to give the confirmation activity instructions and thedevice used to perform the confirmation activity is controlled by thecall back party.

One additional feature aspect of a browser type relay assisted callsystem is referred to as flexible caller ID. In this regard, fortraditional telephone relay service (TRS) services, a relay userconnects to a relay service provider. The service provider then dials toa called party on another line. To provide functional equivalence of aregular telephone call, the relay service equipment forwards theidentity information of the in-bound caller on to the out-boundconnection. The result is that when the dialed party looks at a callerID display on their telephone, the dialed party sees that they arereceiving a call from the in-bound caller, not the relay service. Thisis referred to as “True Caller ID”. Here, the network caller ID systemis not designed to show more than a single identity. Therefore, thecaller ID service is limited to showing that a call is from an in-boundcaller or showing that the call is from a relay service but not both.

The browser type relay assisted call system described above is differentthan a traditional relay system. To this end, as described above, in abrowser type relay assisted call system, a system user receives noticeon their computer/browser screen (see again 14 in FIG. 5) that they arereceiving a call. The relay server 54 dials the user on a call backtelephone line for the voice part of a call. While a user's telephonecaller ID is limited to showing a single identity, the inventive systemactually has two displays including the caller ID display and thebrowser screen. As a result, it has been recognized that additionalinformation can be provided to a system user when the user receives acall that can help the user respond to the call in an appropriatefashion.

In one advantageous embodiment, at web page or browser screen can beprovided by relay server 54 that shows the identity of a caller when anincoming call is received. Here, the caller identity would be, in atleast some embodiments, the same information that would be displayed ona local caller ID box if that identity were presented via the user'stelephone. Since the browser screen is connected to the relay server 54,it is implied that the call is via the relay service and is an assistedcall type. In addition, the relay server 54 may be programmed to notforward the identity of the caller on to the caller ID display of thecaption user's telephone but instead to show the call is coming from therelay service provider. Here, by viewing the caller ID display of thetelephone and recognizing that a call is coming through the relayservice provider, the user can make a better informed decision as to howto respond to the incoming call. Thus, the user can see on her telephonecaller ID display that the call is coming through the relay serviceprovider which lets her know that she should look at or open the user'sbrowser to get captions for the call. Once the browser is open, the usercan see who is calling the user.

Should a user receive a call on the user's telephone where the caller IDdisplay does not show that the call is through the relay serviceprovider, she could elect not to answer the call if she does not haveaccess to the browser and she believes that she will need the captionfunction. In this case, she may opt to allow the call to go to voicemail so that a message can be obtained. Thereafter, when she has betteraccess to the browser, she may call back the caller via the relayservice provider.

Here, while the flexible caller ID concept is described generally incontext of a browser, the caller identification feature may beimplemented using an instant messenger client or some other customapplication.

Referring now to FIG. 15, an exemplary method 230 that is consistentwith the flexible caller ID concept is illustrated. Referring also toFIG. 5, at block 232, relay server 54 monitors relay links for secondparty calls to the first party. At block 234, where communication isattempted, control passes to block 236 where the relay 54 obtains thesecond party identification information. At block 238, relay 54 presentsrelay service provider identifying information via the first party'stelephone caller ID display. In addition, simultaneously, relay 54presents second party identifying information via the first partycomputer/browser/IM/custom application, etc. After block 238, the callactivity continues.

Referring to FIG. 16, an exemplary browser web page 210 is illustratedthat includes a caption 212 and second party identifying informationprovided in a field 214. The caption 212 indicates that “A CapTel callis arriving on your telephone from:” where the second party making thecall is identified in field 214. Referring to FIG. 17, an exemplary cellphone 216 is illustrated including a caller ID display 218 where therelay service provider identifying information is provided on display218. Thus, here, the first party receiving the call can identify thatthe call is via the relay service provider by viewing display 218 andcan also identify the second party caller via field 214.

In at least some embodiments it is contemplated that the flexible callerID feature may be provided via a handheld computer that includestelephone service so that the relay service provider identifyinginformation and the calling party information can be provided via asingle display screen. To this end, referring to FIG. 18, an exemplarycomputer/telephone device 220 is illustrated that includes a largedisplay screen 222. Here, the screen 222 may be used to present browserweb pages. In addition, a separate caller ID window 224 may be opened onthe screen 222 to present caller ID information when an incoming call isreceived by device 220. As shown in FIG. 18, when a call comes intodevice 220 and the device is linked to the relay server 54 forcommunication therewith, the calling party identifying information canbe provided via the browser screen as shown at 226 and a separate callerID window 224 can be opened up to identify the relay service provideridentifying information.

Yet one other aspect or feature that may be implemented via the systemsdescribed above is referred to as a call back feature. The call backfeature is particularly useful in the case of emergency calls that arefacilitated via the systems described above.

Making telephone relay service (TRS) 911 calls over an IP network hasbeen a problem for emergency service providers. The problem is that auser is connected to the emergency service provider via IP where the IPdoes not contain or imply any information about the caller's location.In addition, it is common for a 911 service provider to call back 911callers to resolve any issues that develop while servicing a call. Ifthe caller is using IP, the service provider cannot dial back to thecaller because the caller does not have a dialable call back number.Complicating matters further, calls through a relay come from a relaycenter and not from the caller's home. Thus, an emergency serviceprovider would have to know how to dial a relay center and to ask arelay employee to connect the service provider to the IP caller in orderto complete a call back communication.

The call back feature works as follows. When a first party connects to arelay server 54 as described above and dials 911 or some other emergencynumber, the relay server 54 assigns a temporary relay telephone numberto the first party. Here, the period of assignment is selected so as tobe generally sufficient to ensure that emergency service issues areresolved (e.g., 24-28 hours). In addition, the relay server 54 storesthe temporary telephone number along with any other information that canbe used to establish communication with the caller such as, forinstance, an IP address, the caller's telephone number, etc., in acorrelated fashion.

When the relay server 54 contacts the emergency service provider toservice a caller, the relay server 54 provides the temporary telephonenumber corresponding to the caller to the service provider call taker sothat the call taker can, if necessary, use the temporary number tosubsequently communicate with the caller. After the initial call isterminated, if the emergency call taker wants to call back the originalcaller for some reason, the call taker can simply dial the temporarytelephone number that was provided. The temporary telephone numberautomatically routes to the relay server 54 and is automaticallyconverted into information required to re-establish communication withthe original caller. Thus, in at least some embodiments, the convertedinformation includes an IP address and/or a dial back telephone numberfor the original caller and the call is completed to the user.

Referring now to FIG. 19, an exemplary method 252 that is consistentwith at least some embodiments of the call back feature described aboveis illustrated. Referring also to FIG. 5, at block 250, a first partyuses the first party's computer 14 to connect to the relay server 54. Atblock 254, the first party uses computer 14 to specify a targettelephone number to be called. At block 256, relay 54 recognizes thetarget telephone number as an emergency telephone number (e.g., 911). Atblock 258, server 54 assigns a temporary relay telephone number to thefirst party and at block 260 server 54 stores the temporary telephonenumber along with information that can be used to subsequently contactthe first party caller if desired. At block 260 the exemplaryinformation includes the caller's IP address as well as a first partycall back telephone number. At block 262, relay server 54 uses theemergency telephone number to establish a first communication with theemergency service provider and presents the temporary telephone numberto the emergency service provider (e.g., see phone 28 in FIG. 5).Thereafter, the first party and the service provider are linked via therelay 54 and the first communication commences.

In at least some embodiments a relay service may provide both thetemporary telephone number but also the actual telephone number (if theyhave one) of the caller. The actual number may be of value for emergencycall records. The PSAP would then know both the originating number andalso how to call back to a user with communications support of acaptioning relay. In addition, the relay may be able to identify atemporary call back number as an emergency call and to provide a higherpriority for the call, thus providing a more rapid response for thecall.

Referring still to FIGS. 5 and 19, at block 268 server 54 monitors thefirst communication to determine when the first communication wasterminated. Once the first communication is terminated, server 54 startsa timer to time-out a period during which the temporary telephone numberis assigned to the first caller. Here, for instance, an exemplarytime-out period may include 24 hours, 48 hours, one week, etc. At block272, server 54 determines whether or not the timer period is equal tothe time-out period and, once the timer period is equal to the time-outperiod, control passes to block 274 where server 54 unassigns thetemporary telephone number from the first party and the process ends.

At block 272, when the timer period has been shorter than the time-outperiod, control passes to block 278 where relay 54 monitors callsthereto to identify any calls routed to the relay using the firstparty's temporary telephone number. When relay 54 receives a call madeusing the first party's temporary telephone number, control passes toblock 280 where the relay server 54 uses the temporary telephone numberto identify the IP address and call back number for the first party. Atblock 282, relay server 54 establishes communication with the firstparty via the first party's computer and the call back telephone numberand a second communication between the service provider and th originalcaller continues.

In at least some embodiments it is contemplated that, instead ofassigning a temporary relay telephone number only when a targettelephone number is recognized as an emergency number, temporarytelephone number may be assigned to all calls through a relay server.Here, referring again to FIG. 19, process block 256 would be removed.Where temporary numbers are assigned to all relay calls, all emergencycalls would have temporary numbers and the process as described in FIG.19 would be similar.

After a temporary number is unassigned from a party, that temporarynumber can be reused by the relay server with a second party in the samemanner described above.

One additional aspect or feature that may be implemented using thesystem described above is referred to as a call terminating feature. Inthis regard, a relay user can also receive a relay assisted call from asecond party. To this end, a second party can call a relay user bydialing a toll-free relay service number and entering an area code andnumber of a person that they wish to contact via the relay service. Therelay server user the “call to number” as an index into a database andfinds an IP address for the relay user. The relay then sends a messageto the relay user's browser screen (see again 14 in FIG. 5) to alert theuser that a call is arriving and dials the user's call back number(i.e., the user's telephone 72). Here, the call continues the same wayas if the relay user had placed the call to the second party.

In the above case, if a second party attempts to call a relay user andthat user is not currently on-line or is not logged into the relayservice for some reason, a problem exists in that there is currently noway to terminate the call. Here the call terminating feature is providedto allow relay users to select preferential call terminating functions.

To this end, exemplary call terminating features may include, when therelay user is not on-line or is not logged into the relay service,playing an announcement for the calling party informing the callingparty that the party they are attempting to call is not available andthat the call should be tried again at a later time. Another option maybe for the relay server 54 to go ahead and dial the relay user's callback number and allow the user to either log in to the relay service viathe user's browser and answer the call or let the user's telephonevoicemail take a message. One additional option may be to have the relayserver 54 provide a captioned voicemail function by playing a message tothe caller asking the caller to leave a message, recording the voice ofthe caller, transcribing the message into text and, when the relay usernext logs into the relay service, inform the user of the waitingvoicemail. Here, when the relay user wants to retrieve voicemails, therelay server 54 would play the voice and text back to the user. Thislater method can operate as an accessible form of voicemail for hearingimpaired relay users. In at least some embodiments the relay user canselect how they prefer to handle calls when they are not logged into theservice and their preferences are kept as part of a user's profile whichcan be changed whenever the user wishes.

Referring now to FIG. 20, an exemplary method 290 for setting up orselecting one or a plurality of call terminating features isillustrated. At block 291 a set up process is started wherein a relayuser is provided a browser web page via the user's computer 14 thatincludes a plurality of possible terminating activities that may beperformed when a communication is attempted with the user and the useris not connected to the relay. Here, for instance, the possibleterminating activities may include providing a message indicating that acall cannot be completed, placing a call to the first party's telephoneor facilitating a voice to text transcription enabling the caller toleave a message for the relay user. At block 294 the relay user selectsone of the terminating activities via the browser page and at block 296the terminating activity that was selected by the user is stored in adatabase of user preferences for subsequent use. At block 298 the set upprocess ends.

Referring to FIG. 21, a method 300 for facilitating the call terminatingfeature is illustrated. At block 203, normal relay operation commencesand at block 304, the relay server 54 monitors communications at therelay. At block 306, when a communication is attempted with the firstparty's computer, control passes to block 308. At block 308, relayserver 54 determines whether or not the first party's computer is linkedto the relay. Where the first party's computer is linked to the relay,control passes to block 309 where the call process continues in a mannerdescribed above. However, at block 308, where the first party's computeris not linked to the relay, control passes to the terminating functionthat was selected by the user at block 294 in FIG. 20. Here, threepossible terminating selections are illustrated. A voice message to texttranscription process corresponds to blocks 310, 312, 314 and 316. Tothis end, at block 310, a voicemail message is provided to the callerinstructing the user to leave a voice message. At block 312, where avoice message is left, control passes to block 314 where the voicemessage is transcribed into a text message. At block 316, the textmessage is presented to the first party the next time the first party'scomputer is linked to the relay.

Referring still to FIG. 21, terminating feature 318 simply provides amessage that indicates that a call cannot be completed and that thecaller should try to call back at a later time. Terminating feature 320dials the relay user's telephone even when the relay user's computer isnot linked to the relay server 54 and allows a normal voicemail to beleft or can be used to provide a prompt to the relay user to log intothe service to complete an assisted call.

One additional aspect or feature that may be implemented using thesystems described above is referred to as a Lock Call Back Feature. Asdescribed above, in at least some embodiments a relay user can easilyspecify any call back number to be used to facilitate a relay assistedcall. The ability of a user to specify a call back number isparticularly useful because it allows a user to use virtually anycomputer browser and telephone to place a relay assisted call.

While the ability to specify a call back number is advantageous forvarious reasons, there is some concern that relay users may provideothers with their log in information and share their service privileges.The Lock Call Back Feature enables a relay service provider to lock acall back telephone number so that it cannot be changed easily by arelay user. When implemented, this feature essentially limits a relayuser so that the user can only use a single telephone and a singletelephone number associated therewith. Here the user cannot providetheir log in information to another party unless that other party isusing the same telephone as the user.

It is specifically intended that the present invention not be limited tothe embodiments and illustrations contained herein, but include modifiedforms of those embodiments including portions of the embodiments andcombinations of elements of different embodiments as come within thescope of the following claims.

We claim:
 1. A method to transcribe communications, the methodcomprising: obtaining, at a first communication device, firstcommunication data during a communication session between the firstcommunication device and a second communication device; transmitting thefirst communication data to the second communication device by way of adevice that is locally coupled with the first communication device;receiving, at the first communication device, second communication datafrom the second communication device through the device; transmittingthe second communication data to a remote transcription system;receiving, at the first communication device, transcription data fromthe remote transcription system, the transcription data corresponding toa transcription of the second communication data, the transcriptiongenerated by the remote transcription system; and presenting, by thefirst communication device, the transcription of the secondcommunication data.
 2. The method of claim 1, wherein the secondcommunication data includes audio data or the audio data and video data.3. The method of claim 1, further comprising establishing localcommunication between a first communication device and the device in async mode.
 4. The method of claim 1, wherein the transcription data isreceived at the first communication device after routing thetranscription data through the device.
 5. The method of claim 1, whereintransmitting the second communication data to the remote transcriptionsystem includes routing the second communication data through thedevice.
 6. The method of claim 1, wherein the second communication datais transmitted to the remote transcription system without routing thesecond communication data through the device.
 7. A system comprising:one or more processors; a microphone coupled to the one or moreprocessors, the microphone configured to obtain audio of a user of thesystem and to generate first communication data based on the audio; oneor more computer-readable media coupled to the one or more processors,the one or more computer-readable media configured to store instructionsthat when executed by the one or more processors cause the system toperform operations, the operation comprising: establish local wirelesscommunication between the system and a relay device; obtain anindication of a request for a communication session originating from acommunication device by way of the relay device; direct the firstcommunication data to the communication device by way of the relaydevice, the first communication data being part of the communicationsession between the system and the communication device; obtain, by wayof the relay device, second communication data of the communicationsession originating from the communication device; transmit the secondcommunication data to a remote transcription system; and obtaintranscription data originating from the remote transcription system, thetranscription data corresponding to a transcription of the secondcommunication data, the transcription generated by the remotetranscription system; a speaker coupled to the one or more processorsand configured to broadcast the second communication data; and anelectronic display coupled to the one or more processors and configuredto present the transcription of the second communication data.
 8. Thesystem of claim 7, wherein the second communication data includes audiodata or the audio data and video data.
 9. The system of claim 8, whereinwhen the second communication data includes the audio data and the videodata, only the video data used to provide a video representation via thesecond communication device.
 10. The system of claim 7, furthercomprising disabling communication between the relay device and thesystem while communication between the communication device and therelay device is maintained.
 11. The system of claim 7, wherein theindication of the request for the communication session is automaticallyrouted by the relay device to the system without input from the user.12. A system comprising: means for obtaining first audio data; means fortransmitting the first audio data to a communication device by way of arelay device, the first audio data being part of a communication sessionbetween the system and the communication device; means for receiving, byway of the relay device, second communication data of the communicationsession originating from the communication device; means fortransmitting the second communication data to a remote transcriptionsystem; means for receiving transcription data originating from theremote transcription system, the transcription data corresponding to atranscription of the second communication data, the transcriptiongenerated by the remote transcription system; and means for presentingthe transcription of the second communication data.
 13. The system ofclaim 12, further comprising means for receiving an indication of arequest for the communication session originating from the communicationdevice by way of the relay device.
 14. A method to transcribecommunications, the method comprising: obtaining, at a firstcommunication device, first communication data during a communicationsession between the first communication device and a secondcommunication device; transmitting the first communication data to thesecond communication device by way of a device that is locally coupledwith the first communication device; receiving, at the firstcommunication device, second communication data from the secondcommunication device through the device; transmitting, by the device,the second communication data to a remote transcription system;receiving, at the device, transcription data from the remotetranscription system, the transcription data corresponding to atranscription of the second communication data, the transcriptiongenerated by the remote transcription system; and presenting thetranscription of the second communication data.
 15. The method of claim14, wherein the second communication data includes audio data.
 16. Themethod of claim 15, wherein when the second communication data includesthe audio data and video data.
 17. The method of claim 14, furthercomprising routing, by the device, the transcription data to the firstcommunication device, wherein the first communication device presentsthe transcription of the second communication data.
 18. A method totranscribe communications, the method comprising: establishing acommunication session between a first device and a remote communicationdevice; obtaining, by the first device, first communication data;transmitting, by the first device, the first communication data to theremote communication device during the communication session; receiving,at the first device, second communication data from the remotecommunication device; transmitting, from the first device, the secondcommunication data to a remote transcription system; receiving, at thefirst device, transcription data from the remote transcription system,the transcription data corresponding to a transcription, generated bythe remote transcription system, of the second communication data;transmitting the transcription data to a second device that is locallycoupled with the first device; and presenting, by the second device, thetranscription of the second communication data.
 19. The method of claim18, wherein the first device obtains the first communication data fromthe second device.
 20. The method of claim 18, further comprising:transmitting, from the first device, the second communication data tothe second device; and presenting, by the second device, the secondcommunication data.
 21. The method of claim 18, wherein the first deviceand the second device are locally coupled before the communicationsession is established between the first device and the remotecommunication device.
 22. The method of claim 18, wherein the secondcommunication data includes audio data or the audio data and video data.23. A method to transcribe communications, the method comprising:establishing a local communication between a first device and a seconddevice; establishing a communication session between the first deviceand a remote communication device; receiving, at the first device,communication data from the remote communication device; transmittingthe communication data to a remote transcription system; transmittingthe communication data to the second device; presenting, by the seconddevice, the communication data; and receiving, at the first device,transcription data from the remote transcription system, thetranscription data corresponding to a transcription, generated by theremote transcription system, of the communication data.
 24. The methodof claim 23, wherein the communication data includes audio data or theaudio data and video data.
 25. The method of claim 23, furthercomprising presenting the transcription of the communication data. 26.The method of claim 25, wherein the transcription of the communicationdata is presented by the second device.
 27. The method of claim 23,wherein the communication session between the first device and theremote communication device is established after the local communicationbetween a first device and a second device is established.
 28. A methodto transcribe communications, the method comprising: obtaining, by afirst device, first communication data; transmitting, by the firstdevice, the first communication data to a communication device during acommunication session between the first device and the communicationdevice; receiving, at the first device, second communication data fromthe communication device, the second communication data also beingprovided to a remote transcription system; receiving, at a seconddevice, transcription data from the remote transcription system, thetranscription data corresponding to a transcription, generated by theremote transcription system, of the second communication data;obtaining, at the first device, the transcription data from the seconddevice, the second device locally coupled with the first device; andpresenting, by the first device, the transcription of the secondcommunication data.
 29. The method of claim 28, wherein the secondcommunication data includes audio data.
 30. The method of claim 28,wherein the first device is a caption device.
 31. The method of claim28, further comprising providing, by the first device, the secondcommunication data to the remote transcription system.
 32. The method ofclaim 31, wherein the second communication data is provided by the firstdevice to the remote transcription system by way of the second device.33. The method of claim 28, wherein the communication session betweenthe first device and the communication device occurs over a publicswitched telephone network.
 34. The method of claim 28, wherein thesecond device is locally coupled with the first device over a localwireless network.
 35. A system comprising: one or more processors; amicrophone coupled to the one or more processors, the microphoneconfigured to obtain audio of a user of the system and to generate firstcommunication data based on the audio; one or more computer-readablemedia coupled to the one or more processors, the one or morecomputer-readable media configured to store instructions that whenexecuted by the one or more processors cause the system to performoperations, the operations comprising: establish local wirelesscommunication between the system and a relay device; direct the firstcommunication data to a communication device, the first communicationdata being part of a communication session between the system and thecommunication device; obtain second communication data of thecommunication session originating from the communication device, thesecond communication data also being provided to a remote transcriptionsystem; and obtain, by way of the relay device, transcription dataoriginating from the remote transcription system, the transcription datacorresponding to a transcription of the second communication data, thetranscription generated by the remote transcription system and providedto the relay device; a speaker coupled to the one or more processors andconfigured to broadcast the second communication data; and an electronicdisplay coupled to the one or more processors and configured to presentthe transcription of the second communication data.
 36. The system ofclaim 35, wherein the second communication data includes audio data. 37.The system of claim 35, wherein the operations further comprise providethe second communication data to the remote transcription system. 38.The system of claim 37, wherein the second communication data isprovided to the remote transcription system by way of the relay device.39. The system of claim 35, wherein the communication session betweenthe system and the communication device occurs over a public switchedtelephone network.
 40. At least one non-transitory computer-readablemedia configured to store instructions that, when executed by a system,cause the system to perform operations, the operations comprising:obtain, by the system, first communication data based on audio from auser of the system; establish local wireless communication between thesystem and a relay device; direct the first communication data to acommunication device, the first communication data being part of acommunication session between the system and the communication device;obtain second communication data of the communication sessionoriginating from the communication device, the second communication dataalso being provided to a remote transcription system; obtain, by way ofthe relay device, transcription data originating from the remotetranscription system, the transcription data corresponding to atranscription of the second communication data, the transcriptiongenerated by the remote transcription system and provided to the relaydevice; and present, by the system, the transcription of the secondcommunication data.
 41. The non-transitory computer-readable media ofclaim 40, wherein the system is a caption device.
 42. The non-transitorycomputer-readable media of claim 40, wherein the operations furthercomprise provide the second communication data to the remotetranscription system.
 43. The non-transitory computer-readable media ofclaim 42, wherein the second communication data is provided to theremote transcription system by way of the relay device.
 44. Thenon-transitory computer-readable media of claim 40, wherein thecommunication session between the system and the communication deviceoccurs over a public switched telephone network.